{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "# Ascend910与GPU推理\n",
    "\n",
    "<a href=\"https://gitee.com/mindspore/docs/blob/master/tutorials/source_zh_cn/middleclass/inference/ascend910_inference.ipynb\" target=\"_blank\"><img src=\"https://gitee.com/mindspore/docs/raw/master/resource/_static/logo_source.png\"></a>\n",
    "\n",
    "\n",
    "本文将介绍如何在Ascend910和GPU硬件环境中，利用MindIR和Checkpoint执行推理。MindIR是MindSpore的统一模型文件，同时存储了网络结构和权重参数值，定义了可扩展的图结构以及算子的IR表示，消除了不同后端的模型差异，一般用于跨硬件平台执行推理任务。Checkpoint是训练参数，采用了Protocol Buffers格式，一般用于训练任务中断后恢复训练，或训练后的微调（Fine Tune）任务。\n",
    "\n",
    "下面将针对这两种情况，介绍如何使用MindSpore进行单卡推理。\n",
    "\n",
    "## 使用checkpoint格式文件单卡推理\n",
    "\n",
    "### 使用本地模型推理\n",
    "\n",
    "用户可以通过`load_checkpoint`和`load_param_into_net`接口从本地加载模型与参数，传入验证数据集后使用`model.eval`即可进行模型验证，使用`model.predict`可进行模型推理。在这里我们下载MindSpore Hub中已经预训练好的LeNet和MINIST数据集进行推理演示："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "!mkdir -p ./datasets/MNIST_Data/test\n",
    "!wget -NP ./datasets/MNIST_Data/test https://mindspore-website.obs.myhuaweicloud.com/notebook/datasets/mnist/t10k-labels-idx1-ubyte\n",
    "!wget -NP ./datasets/MNIST_Data/test https://mindspore-website.obs.myhuaweicloud.com/notebook/datasets/mnist/t10k-images-idx3-ubyte\n",
    "!tree ./datasets/MNIST_Data\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "!mkdir -p ./checkpoint\n",
    "!wget -NP ./checkpoint https://download.mindspore.cn/model_zoo/r1.1/lenet_ascend_v111_offical_cv_mnist_bs32_acc98/lenet_ascend_v111_offical_cv_mnist_bs32_acc98.ckpt\n",
    "!tree ./checkpoint"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "配置运行所需信息，进行推理的数据处理："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import argparse\n",
    "from mindspore import context\n",
    "\n",
    "import mindspore.dataset as ds\n",
    "import mindspore.dataset.transforms.c_transforms as C\n",
    "import mindspore.dataset.vision.c_transforms as CV\n",
    "from mindspore.dataset.vision import Inter\n",
    "from mindspore import dtype as mstype\n",
    "\n",
    "parser = argparse.ArgumentParser(description='MindSpore Inference Example')\n",
    "parser.add_argument('--device_target', type=str, default=\"GPU\", choices=['Ascend', 'GPU'])\n",
    "\n",
    "args = parser.parse_known_args()[0]\n",
    "context.set_context(mode=context.GRAPH_MODE, device_target=args.device_target)\n",
    "\n",
    "def create_dataset(data_path, batch_size=32, repeat_size=1,\n",
    "                   num_parallel_workers=1):\n",
    "    # 定义数据集\n",
    "    mnist_ds = ds.MnistDataset(data_path)\n",
    "    resize_height, resize_width = 32, 32\n",
    "    rescale = 1.0 / 255.0\n",
    "    shift = 0.0\n",
    "    rescale_nml = 1 / 0.3081\n",
    "    shift_nml = -1 * 0.1307 / 0.3081\n",
    "\n",
    "    # 定义所需要操作的map映射\n",
    "    resize_op = CV.Resize((resize_height, resize_width), interpolation=Inter.LINEAR)\n",
    "    rescale_nml_op = CV.Rescale(rescale_nml, shift_nml)\n",
    "    rescale_op = CV.Rescale(rescale, shift)\n",
    "    hwc2chw_op = CV.HWC2CHW()\n",
    "    type_cast_op = C.TypeCast(mstype.int32)\n",
    "\n",
    "    # 使用map映射函数，将数据操作应用到数据集\n",
    "    mnist_ds = mnist_ds.map(operations=type_cast_op, input_columns=\"label\", num_parallel_workers=num_parallel_workers)\n",
    "    mnist_ds = mnist_ds.map(operations=resize_op, input_columns=\"image\", num_parallel_workers=num_parallel_workers)\n",
    "    mnist_ds = mnist_ds.map(operations=rescale_op, input_columns=\"image\", num_parallel_workers=num_parallel_workers)\n",
    "    mnist_ds = mnist_ds.map(operations=rescale_nml_op, input_columns=\"image\", num_parallel_workers=num_parallel_workers)\n",
    "    mnist_ds = mnist_ds.map(operations=hwc2chw_op, input_columns=\"image\", num_parallel_workers=num_parallel_workers)\n",
    "\n",
    "    # 进行shuffle、batch操作\n",
    "    buffer_size = 10000\n",
    "    mnist_ds = mnist_ds.shuffle(buffer_size=buffer_size)\n",
    "    mnist_ds = mnist_ds.batch(batch_size, drop_remainder=True)\n",
    "\n",
    "    return mnist_ds\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "创建LeNet模型："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "import mindspore.nn as nn\n",
    "from mindspore.common.initializer import Normal\n",
    "\n",
    "class LeNet5(nn.Cell):\n",
    "    \"\"\"\n",
    "    Lenet网络结构\n",
    "    \"\"\"\n",
    "    def __init__(self, num_class=10, num_channel=1):\n",
    "        super(LeNet5, self).__init__()\n",
    "        # 定义所需要的运算\n",
    "        self.conv1 = nn.Conv2d(num_channel, 6, 5, pad_mode='valid')\n",
    "        self.conv2 = nn.Conv2d(6, 16, 5, pad_mode='valid')\n",
    "        self.fc1 = nn.Dense(16 * 5 * 5, 120, weight_init=Normal(0.02))\n",
    "        self.fc2 = nn.Dense(120, 84, weight_init=Normal(0.02))\n",
    "        self.fc3 = nn.Dense(84, num_class, weight_init=Normal(0.02))\n",
    "        self.relu = nn.ReLU()\n",
    "        self.max_pool2d = nn.MaxPool2d(kernel_size=2, stride=2)\n",
    "        self.flatten = nn.Flatten()\n",
    "\n",
    "    def construct(self, x):\n",
    "        # 使用定义好的运算构建前向网络\n",
    "        x = self.conv1(x)\n",
    "        x = self.relu(x)\n",
    "        x = self.max_pool2d(x)\n",
    "        x = self.conv2(x)\n",
    "        x = self.relu(x)\n",
    "        x = self.max_pool2d(x)\n",
    "        x = self.flatten(x)\n",
    "        x = self.fc1(x)\n",
    "        x = self.relu(x)\n",
    "        x = self.fc2(x)\n",
    "        x = self.relu(x)\n",
    "        x = self.fc3(x)\n",
    "        return x\n",
    "\n",
    "# 实例化网络\n",
    "net = LeNet5()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "在推理进行前，需要使用`load_checkpoint`和`load_param_into_net`接口从本地加载模型与参数。这样一来就可以使用本地模型完成后面的推理过程。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from mindspore import load_checkpoint, load_param_into_net\n",
    "ckpt_file_name = \"./checkpoint/lenet_ascend_v111_offical_cv_mnist_bs32_acc98.ckpt\"\n",
    "param_dict = load_checkpoint(ckpt_file_name)\n",
    "load_param_into_net(net, param_dict)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "设置损失函数与优化器，并调用`model`接口创建对象："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from mindspore.nn import Accuracy\n",
    "from mindspore import Model, Tensor\n",
    "\n",
    "net_loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction=\"mean\")\n",
    "net_opt = nn.Momentum(net.trainable_params(), learning_rate=0.01, momentum=0.9)\n",
    "model = Model(net, net_loss, net_opt, metrics={\"Accuracy\": Accuracy()})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面调用`model.eval`接口执行验证过程："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "============== {'Accuracy': 0.9846754807692307} ==============\n"
     ]
    }
   ],
   "source": [
    "mnist_path = \"./datasets/MNIST_Data/test\"\n",
    "ds_eval = create_dataset(os.path.join(mnist_path))\n",
    "acc = model.eval(ds_eval, dataset_sink_mode=False)\n",
    "print(\"============== {} ==============\".format(acc))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "调用`model.predict`接口执行验证过程，这里选取数据集中的一张图片进行预测："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Predicted: \"6\", Actual: \"6\"\n"
     ]
    }
   ],
   "source": [
    "ds_eval = ds_eval.create_dict_iterator()\n",
    "data = next(ds_eval)\n",
    "\n",
    "# images为测试图片，labels为测试图片的实际分类\n",
    "images = data[\"image\"].asnumpy()\n",
    "labels = data[\"label\"].asnumpy()\n",
    "\n",
    "# 使用函数model.predict预测image对应分类\n",
    "output = model.predict(Tensor(data['image']))\n",
    "predicted = np.argmax(output.asnumpy(), axis=1)\n",
    "\n",
    "# 输出预测分类与实际分类\n",
    "print(f'Predicted: \"{predicted[0]}\", Actual: \"{labels[0]}\"')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 加载MindSpore Hub模型执行推理\n",
    "\n",
    "除了使用`load_checkpoint`和`load_param_into_net`从本地加载模型之外，也可以通过[安装MindSpore Hub](https://gitee.com/mindspore/hub/blob/r1.2/README_CN.md)，通过`mindspore_hub.load`从云端加载模型参数执行推理。\n",
    "\n",
    "之前使用的加载本地模型的方法为：\n",
    "\n",
    "```python\n",
    "from mindspore import load_checkpoint, load_param_into_net\n",
    "ckpt_file_name = \"./checkpoint/lenet_ascend_v111_offical_cv_mnist_bs32_acc98.ckpt\"\n",
    "param_dict = load_checkpoint(ckpt_file_name)\n",
    "load_param_into_net(net, param_dict)\n",
    "```\n",
    "\n",
    "可替换为`mindspore_hub.load`方法：\n",
    "\n",
    "```python\n",
    "import mindspore_hub\n",
    "model_uid = \"mindspore/ascend/1.2/lenet_v1.2\"\n",
    "network = mindspore_hub.load(model_uid)\n",
    "```\n",
    "\n",
    "## 使用C++接口推理MindIR格式文件\n",
    "\n",
    "本小节将介绍如何使用C++接口推理MINDIR格式的模型。完整代码可参考[ascend910_resnet50_preprocess_sample](https://gitee.com/mindspore/docs/tree/master/tutorials/tutorial_code/ascend910_resnet50_preprocess_sample)。\n",
    "\n",
    "### 推理代码介绍\n",
    "\n",
    "完成的推理代码为`main.cc`文件，现在对其中的功能实现进行说明。\n",
    "\n",
    "引用`mindspore`和`mindspore::dataset`的名字空间。\n",
    "\n",
    "```cpp\n",
    "namespace ms = mindspore;\n",
    "namespace ds = mindspore::dataset;\n",
    "```\n",
    "\n",
    "环境初始化，指定硬件为Ascend 910，DeviceID为0：\n",
    "\n",
    "```c++\n",
    "auto context = std::make_shared<ms::Context>();\n",
    "auto ascend910_info = std::make_shared<ms::Ascend910DeviceInfo>();\n",
    "ascend910_info->SetDeviceID(0);\n",
    "context->MutableDeviceInfo().push_back(ascend910_info);\n",
    "```\n",
    "\n",
    "加载模型文件:\n",
    "\n",
    "```cpp\n",
    "// 加载 MindIR 模型\n",
    "ms::Graph graph;\n",
    "ms::Status ret = ms::Serialization::Load(resnet_file, ms::ModelType::kMindIR, &graph);\n",
    "// 进行图编译\n",
    "ms::Model resnet50;\n",
    "ret = resnet50.Build(ms::GraphCell(graph), context);\n",
    "```\n",
    "\n",
    "获取模型所需输入信息：\n",
    "\n",
    "```cpp\n",
    "std::vector<ms::MSTensor> model_inputs = resnet50.GetInputs();\n",
    "```\n",
    "\n",
    "加载图片文件:\n",
    "\n",
    "```cpp\n",
    "ms::MSTensor ReadFile(const std::string &file);\n",
    "auto image = ReadFile(image_file);\n",
    "```\n",
    "\n",
    "图片预处理:\n",
    "\n",
    "```cpp\n",
    "// 对图片进行解码，变为RGB格式，并重设尺寸\n",
    "std::shared_ptr<ds::TensorTransform> decode(new ds::vision::Decode());\n",
    "std::shared_ptr<ds::TensorTransform> resize(new ds::vision::Resize({256}));\n",
    "// 输入归一化\n",
    "std::shared_ptr<ds::TensorTransform> normalize(new ds::vision::Normalize(\n",
    "    {0.485 * 255, 0.456 * 255, 0.406 * 255}, {0.229 * 255, 0.224 * 255, 0.225 * 255}));\n",
    "// 剪裁图片\n",
    "std::shared_ptr<ds::TensorTransform> center_crop(new ds::vision::CenterCrop({224, 224}));\n",
    "// shape (H, W, C) 变为 shape (C, H, W)\n",
    "std::shared_ptr<ds::TensorTransform> hwc2chw(new ds::vision::HWC2CHW());\n",
    "\n",
    "// 定义preprocessor\n",
    "ds::Execute preprocessor({decode, resize, normalize, center_crop, hwc2chw});\n",
    "\n",
    "// 调用函数，获取处理后的图像\n",
    "ret = preprocessor(image, &image);\n",
    "```\n",
    "\n",
    "执行推理:\n",
    "\n",
    "```cpp\n",
    "// 创建输入输出向量\n",
    "std::vector<ms::MSTensor> outputs;\n",
    "std::vector<ms::MSTensor> inputs;\n",
    "\n",
    "inputs.emplace_back(model_inputs[0].Name(), model_inputs[0].DataType(), model_inputs[0].Shape(),\n",
    "                    image.Data().get(), image.DataSize());\n",
    "// 执行推理\n",
    "ret = resnet50.Predict(inputs, &outputs);\n",
    "```\n",
    "\n",
    "获取推理结果:\n",
    "\n",
    "```cpp\n",
    "// 获取推理结果的最大可能性\n",
    "std::cout << \"Image: \" << image_file << \" infer result: \" << GetMax(outputs[0]) << std::endl;\n",
    "```\n",
    "\n",
    "### 构建脚本介绍\n",
    "\n",
    "构建脚本用于构建用户程序，完整代码位于`CMakeLists.txt` ，下面进行解释说明。\n",
    "\n",
    "为编译器添加头文件搜索路径：\n",
    "\n",
    "```cmake\n",
    "option(MINDSPORE_PATH \"mindspore install path\" \"\")\n",
    "include_directories(${MINDSPORE_PATH})\n",
    "include_directories(${MINDSPORE_PATH}/include)\n",
    "```\n",
    "\n",
    "在MindSpore中查找所需动态库：\n",
    "\n",
    "```cmake\n",
    "find_library(MS_LIB libmindspore.so ${MINDSPORE_PATH}/lib)\n",
    "file(GLOB_RECURSE MD_LIB ${MINDSPORE_PATH}/_c_dataengine*)\n",
    "```\n",
    "\n",
    "使用指定的源文件生成目标可执行文件，并为目标文件链接MindSpore库：\n",
    "\n",
    "```cmake\n",
    "add_executable(resnet50_sample main.cc)\n",
    "target_link_libraries(resnet50_sample ${MS_LIB} ${MD_LIB})\n",
    "```\n",
    "\n",
    "\n",
    "### 编译并执行推理代码\n",
    "\n",
    "可选择将实验的脚本下载至Ascend910环境中编译并执行。\n",
    "\n",
    "进入工程目录`ascend910_resnet50_preprocess_sample`，执行`cmake`命令，其中`pip3`需要按照实际情况修改：：\n",
    "\n",
    "```bash\n",
    "cmake . -DMINDSPORE_PATH=`pip3 show mindspore-ascend | grep Location | awk '{print $2\"/mindspore\"}' | xargs realpath`\n",
    "```\n",
    "\n",
    "再执行`make`命令编译即可。\n",
    "\n",
    "```bash\n",
    "make\n",
    "```\n",
    "\n",
    "编译成功后，会获得`resnet50_sample`可执行文件。选取几张图片，输入执行命令即可获取推理结果：\n",
    "\n",
    "```bash\n",
    "./resnet50_sample\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
